System and method for determining vehicle data set familiarity

ABSTRACT

The present disclosure relates to systems, devices and methods for identifying objects and scenarios that have not been trained or are unidentifiable to vehicle perception sensors or vehicle assistive driving systems. Embodiments are directed to using a trained vehicle data set to identify target objects in vehicle sensor data. In one embodiment, a process is provided that includes running a scene detection operation on vehicle to derive a vector of target object attributes of the vehicle sensor data and generating a vector representation for the scene detection operation and the attributes of the vehicle sensor data. The vector representation compared to a familiarity vector to represent effectiveness of the scene detection operation. In addition, the vector representation can be scored to identify one or more target objects or significant scenarios, including unidentifiable objects and/or driving scenes, scenarios for reporting.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is concurrently filed with U.S. application Ser. No.______, entitled SYSTEM AND METHOD FOR EVALUATING A TRAINED VEHICLE DATASET FAMILIARITY OF A DRIVER ASSISTANCE SYSTEM filed on ______, thedisclosure of which is hereby expressly incorporated by reference.

FIELD

The present disclosure relates to systems, methods and devices forassistive and autonomous driving and more particularly to identifyingsignificant data.

BACKGROUND

Vehicle systems are being developed to provide assistance with operationof a vehicle. The systems can include providing feedback and vehiclecontrol. There exists a need for systems and processes to accuratelyidentify data that is important to vehicle detection systems. Many ofthe existing systems collect large amounts of data which is hardutilize. By way of example, a vehicle system including multiple sensorsmay generate a vast amount of data during operation. Because existingsystems typically generate and store data indiscriminately, there is asignificant cost associated storage and maintaining data. In addition toprocessing, these systems require large scale data storage capabilities.Drawbacks of these systems can include not being able to handle dataefficiently, cost for storing data, and difficulty in identifying dataof relevance. Existing systems do not provide configurations tointerpret significant data. As a result, existing systems may behampered by data that is stored and not useful. There exists need toimprove configurations of vehicle systems for assisted driving and toidentify data of significance.

BRIEF SUMMARY OF THE EMBODIMENTS

Disclosed and claimed herein are methods, devices and systems forvehicle perception system operations. One embodiment is directed toidentifying significant scenario data by a control unit of a vehicle.The method includes receiving, by a control unit, vehicle sensor datacaptured by at least one sensor of the vehicle. The vehicle sensor datais generated by a driver assistance system of a vehicle. The method alsoincludes running, by the control unit, a scene detection operation onthe vehicle sensor data using a trained vehicle data set to identifytarget object attributes of the vehicle sensor data. The method alsoincludes generating, by the control unit, a vector representation forthe scene detection operation and the attributes of the vehicle sensordata, wherein the vector representation is a representation ofeffectiveness of the scene detection operation in identifying targetobject attributes of the vehicle sensor data. The method also includesidentifying, by the control unit, significant scenario data based on thevector representation, wherein the significant scenario identifies atleast one target object of the vehicle sensor data.

In one embodiment, the vehicle sensor data includes at least one ofimage, radar, and LiDAR data for a detection zone of the driverassistance system of the vehicle.

In one embodiment, running the scene detection operation on the vehiclesensor data generates an annotated data set for target objects in realtime based on the attributes of the trained vehicle data set, thetrained vehicle data set providing a plurality of object types andobject attributes.

In one embodiment, generating the vector representation includesperforming a clustering operation for target objects of the vehiclesensor data using the trained vehicle data set to generate a vector datamodel for the vehicle sensor data, the vector data model characterizingability of the trained vehicle set to perceive target objects of thevehicle sensor data.

In one embodiment, identifying the significant scenario includesdetermining that a target object is an unidentified object.

In one embodiment, identifying the significant scenario includesdetermining that at least one of the trained vehicle data attributes areunable to classify a target object.

In one embodiment, identifying the significant scenario includesdetermining familiarity of a target object relative to the trained dataset based on at least one of the number of target objects,classification of target objects, size and shape of target object,object type, and object color.

In one embodiment, identifying the significant scenario includesdetermining at least one vehicle operation characteristic as anattribute relative to identification of a target object in at least oneof a driver assistance system and autonomous driving system.

In one embodiment, identifying the significant scenario includesdetermining familiarity of a current scene the vehicle drives throughrelative to the vehicle trained data set for a driving condition.

In one embodiment, the method further includes scoring, by the controlunit, the vector representation on the ability of the scene detectionoperation to perceive target object attributes of the vehicle sensordata using the trained vehicle data set, and wherein the significantscenario is identified data based on a score of the vectorrepresentation below a predetermined threshold.

In one embodiment, the method further includes outputting the at leastone target object of the vehicle sensor data.

Another embodiment is directed to a vehicle control unit including aninput configured to receive vehicle sensor data, and a control unitcoupled to the input. The control unit is configured to receive vehiclesensor data captured by at least one sensor of the vehicle, the vehiclesensor data generated by a driver assistance system of a vehicle. Thecontrol nit is also configured to run a scene detection operation on thevehicle sensor data using a trained vehicle data set to identify targetobject attributes of the vehicle sensor data. The control unit is alsoconfigured to generate a vector representation for the scene detectionoperation and the attributes of the vehicle sensor data, wherein thevector representation is a representation of effectiveness of the scenedetection operation in identifying target object attributes of thevehicle sensor data. The control unit is also configured to identifysignificant scenario data based on the vector representation, whereinthe significant scenario identifies at least one target object of thevehicle sensor data.

Other aspects, features, and techniques will be apparent to one skilledin the relevant art in view of the following detailed description of theembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The features, objects, and advantages of the present disclosure willbecome more apparent from the detailed description set forth below whentaken in conjunction with the drawings in which like referencecharacters identify correspondingly throughout and wherein:

FIGS. 1A-1B depict graphical representations of scenario data accordingto one or more embodiments;

FIG. 2 depicts a process for control unit operation according to one ormore embodiments;

FIG. 3 depicts a graphical representation of a vehicle control unitaccording to one or more embodiments;

FIG. 4 depicts a graphical representation of a trained vehicle data setaccording to one or more embodiments;

FIG. 5 depicts a graphical representation of control unit operationsaccording to one or more embodiments;

FIG. 6 depicts a graphical representation of operations relative to aknown data set according to one or more embodiments; and

FIG. 7 depicts a graphical representation of object attributes accordingto one or more embodiments.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS Overview andTerminology

One aspect of the disclosure is directed to identifying significant datadetected by a vehicle system, and in particular assistive or autonomousdriving systems. In one embodiment, identifying significant dataincludes identifying data that is unfamiliar to an existingconfiguration for detecting objects. Otherwise stated, systems,configurations and processes are provided to identify at least one of anobject, object attribute, and scenario perceived by a vehicle systemwhich is underrepresented or brand new in the trained data set.Identifying significant and/or underrepresented data is one exemplaryuse of the configurations and operations discussed herein. Embodimentsare also directed to detection and characterization of anomalies invehicle data sets. As used herein, one or more anomalies may bedetermined with respect to vehicle sensor data using a trained vehicledata set. In some instances, a trained vehicle data set is not able toidentify or classify a detected object. By way of example, one or morevehicle sensors may provide output identifying the presence and/orlocation of an object where the object may not match or be representedby an object type of the trained data set. In other embodiments, vehiclesensor data may conflict with respect to a detected object. Failure toidentify or classify an object may be based on insufficient data orparameters of a trained data set used by the vehicle to perceive ascene.

As used herein, assistive driving may refer to at least one of providingan indication, capturing data, controlling operation of a vehicle,activating a vehicle system and vehicle control operations in generalfor driver and/or driverless controlled vehicles. Assistive driving mayrelate to one or more functions of an advanced driver assistance system(ADAS) or autonomous driving (AD) system. Assistive driving operationsmay include lane change assist, parking assist, braking control, etc.Assistive driving may also refer to systems and operations forautonomous driving systems and vehicles. An assistive driving system canrelate to a vehicle system including at least one sensor to detectobjects or vehicle operating conditions, a control unit and one or morevehicle systems to control the vehicle. Object and vehicle data may bedetermined from at least one sensor, including but not limited to animage sensor (e.g., video), radar sensor, LiDAR sensor,acceleration/motion sensor and vehicle sensors (e.g., wheel speedsensor, tire pressure monitoring system (TPMS) sensor, ABS sensor,yaw/pitch sensor, stability control sensor, etc.). According to oneembodiment, assistive driving systems may operate using baseline data,such as a trained data set to detect target objects (e.g., othervehicles, lane markers, barriers, etc.), driving conditions (e.g., abraking condition, vehicle sliding, distance to other vehicles/objects,etc.) and/or scenarios. The baseline data may include one or morerecognized object types and object attributes. The baseline data canalso include scenario data for objects and object arrangement. Forexample, patterns of lane markers may be detected to signal a lanemerger or bend in a road. As will be discussed herein, objects, objectattributes, and scene data may be part of a scenario which may beidentified and characterized. Systems, methods and configurationsdescribed herein, can include vehicle systems for real-world deploymentin various conditions or scenes (e.g., day, night, low-light, weatherrelated, etc.). In addition, systems and methods may be performed duringrun time (e.g., while the program is being executed) for in-vehicle use.

According to another aspect of the disclosure, embodiments are directedto event capture and operations for analyzing vehicle sensor data. Oneissue for a vehicle system that captures large amounts of data isidentifying data that is significant. Embodiments are directed toincorporating the significant data into a trained vehicle data set. Dataof significance may be used to determine capture events and datasegments, enhance the similarity of a baseline data set to the realworld, and determining whether scene data detected overall is valid.

In one embodiment, processes include generating a vector representationof trained data set or annotated baseline data. In other embodiments,processes include generating a vector representation of vehicle sensordata. Vector representations may be based on model data of the systemand may be used to determine effectiveness of the system in identifyingobjects using a trained data set and/or effectiveness of the traineddata set. Generating a vector representation may include performing oneor more operations by a vehicle controller. In one embodiment, vectorrepresentation allows for an anomaly vector to be determined. Theanomaly vector may be used to identify objects or scenarios that are ofsignificance to the system.

In one embodiment, processes include comparing a vector representationof trained data set, such as a familiarity vector, to a vector derivedfor target object attributes detected from at least one perceptionsensor of a vehicle.

According to one embodiment vector representations may be employed fordetermining a significant scenario based on vector representations ofobject type, object attributes, object patterns, and scenes. By way ofexample, vector representations of the object type may identify anobject that does not match or cannot be identified with parameters ofthe data set for identifying an object. In one embodiment, objectattributes may be significant, where a target object is detected and isclassified as an object. The target object may include one or moreattributes that are identified and not adequately handled by a controldevice, accordingly the one or more attributes may be significant. Inanother embodiment, vector representations of object patterns, such aslane markers, parking situation patterns, braking patterns for stop andgo traffic, etc., may be handled by a vector representation for thepattern. Situations which differ from patterns of the data set, ordriving situations including a pattern and then a divergence from thepattern may be identified as significant. One or more vectorrepresentations may be derived for target object attributes from vehiclesensor data. Vector representations may be generated for a scene basedon one or more parameters. For a driving scene that is relative to otherobjects, such as other vehicles, movement of the other vehicles and thevehicle including a control unit may be detected to identify significantscenarios for the vehicle including a control unit, such as a trafficpattern the vehicle is not properly trained to handle. Alternatively,scenarios may relate to driving conditions based on weather (e.g., rain,snow, fog, etc.), road condition (e.g., paved, unpaved, low traction,etc.), lighting conditions (e.g., low light, and operating scenarios fora vehicle.

According to one embodiment, identifying a significant scenario includesidentifying at least one target object. The target object may besignificant if vehicle trained data is not sufficient to characterize ordetermine the type of object detected. A target object may besignificant if one or more attributes of the trained vehicle data doesnot adequately identify an object. Significant scenarios may be detectedbased on vehicle operation.

According to another embodiment, a significant scenario may beidentified by comparing a vector representation for a scene detectionoperation with a familiarity vector of a trained data set, such as avehicle trained data set.

According to one embodiment, identifying a significant scenario includesdetecting one or more vehicle operating conditions. Vehicle operatingconditions may be determined based on vehicle data, such as loss oftraction or rapid deceleration events (e.g., braking event) usingvehicle data. In other embodiments, vehicle operating conditions may berelative to identified objects. By way of example, detection of anobject, such as a vehicle traveling in the same direction or in front ofa vehicle, may be used to for detection of scenario.

According to one embodiment, a control unit of a vehicle is configuredto identify significant scenario data. Processes and deviceconfigurations are provided to identify significant scenario datadetected from one or more vehicle sensors. As such, significant vehicledata may be detected during runtime. In addition, vehicle training dataand the ability of a vehicle system to detect objects and drivingconfigurations may be evaluated. In one embodiment a vehicle controlunit includes at least one input configured to receive vehicle sensordata, and a control unit coupled to the input. The control unit may bepart of a vehicle control system, such as an assistive driving unit orautonomous driving module. The control unit may be configured to receivevehicle sensor data and perform one or more operations to evaluate ascene associated with a vehicle and/or one or more operatingcharacteristics. As will be discussed in more detail below, the controlunit may employ a vector representation to process attributes of thevehicle sensor data.

Processes and device configurations described herein can avoid reportingdata that is easily handled by the system. Reporting significant data,such as an object of significance, is preferable to limit the data thathas to be processed to evaluate a vehicle trained data set. Theprocesses and device configurations described herein allow for a vehiclecontrol unit to identify significant or relevant objects and capture ofdata for further analysis. These operations may allow for a vehicleassistance system to continually approach a desired data set ofparameters while limiting and/or eliminating the need for manual review.

As used herein, the terms “a” or “an” shall mean one or more than one.The term “plurality” shall mean two or more than two. The term “another”is defined as a second or more. The terms “including” and/or “having”are open ended (e.g., comprising). The term “or” as used herein is to beinterpreted as inclusive or meaning any one or any combination.Therefore, “A, B or C” means “any of the following: A; B; C; A and B; Aand C; B and C; A, B and C”. An exception to this definition will occuronly when a combination of elements, functions, steps or acts are insome way inherently mutually exclusive.

Reference throughout this document to “one embodiment,” “certainembodiments,” “an embodiment,” or similar term means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment. Thus, the appearancesof such phrases in various places throughout this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics may be combined inany suitable manner on one or more embodiments without limitation.

EXEMPLARY EMBODIMENTS

FIGS. 1A-1B depict graphical representations of scenario data accordingto one or more embodiments. According to one embodiment, systems,configurations and processes described herein are directed to vehiclesystems. FIGS. 1A-1B depict graphical representations of an exemplaryscenario for a scene 100 associated with vehicle 105 and a plurality ofobjects. Scene 100 may relate to a particular location or period oftime, wherein the presence of objects and/or one of more vehiclecharacteristics may allow for a sensor of vehicle 105 to detect one ormore objects. According to one embodiment, vehicle 105 may include oneor more sensors to detect objects based on vehicle trained data. By wayof example, for objects present in scene 100, vehicle 105 is configuredto interpret vehicle sensor data to detect one or more of the objects.In certain embodiments, vehicle sensor data may be related to one deviceor one type of device, such as an image sensor (e.g., camera). As such,one or more target objects may be identified by vehicle 105 usingvehicle trained data from sensed image data. According to otherembodiment, vehicle 105 may include a plurality of sensing devicesgenerating multiple types of sensor output, including one or more ofimage, proximity sensing, radar, and LiDAR.

Vehicle 105 may be configured to detect and classify a plurality ofobject types and scenarios, such as braking event or lane shift, usingtrained vehicle data. Embodiments discussed herein allow for assessingwhen vehicle trained data sets do not sufficiently allow for detectionor handling of target objects. By way of example, vehicle 105 may detectan object, but not be able to identify or determine how to controlvehicle 105 in response to the object. Other embodiments allow forassessing the vehicle familiarity with objects using a trained data set.In that fashion, while the embodiments may employ specific sensor typesand/or target object attributes, the systems, processes andconfiguration discussed herein allow for detecting anomalies with one ormore different sensor configurations or vehicle training data sets.Moreover, the principles of the disclosure can be applied to differentsensor configurations of a vehicle.

FIG. 1A depicts an exemplary scene including objects which may bedetected and identified by one or more units, such as a control unit ofvehicle 105. In FIG. 1A, scene 100 includes vehicle 105 having one ormore detection areas shown as 110 and 111, the detection areas generallyreferring to a forward direction of travel for detection area 110 and abackward facing direction for detection area 111 of the vehicle 105.Detection areas 110 and 111 are shown relative to one or more directionsof vehicle 105 that may be assessed and or accounted for in the trainedvehicle data set. According to one embodiment, one or more sensors ofvehicle 105 can detect objects relative to the detection areas of thevehicle, including forward, back, and lateral direction areas of thevehicle. Scene 100 may relate to an exemplary scenario, or type of sceneof interest to the vehicle. In addition, each scene detected by vehicle105 may be associated with one or more of the type of object, number ofobjects, location of objects, movement of objects, etc. According toanother embodiment, each scene may be associated with at least one ofthe operating characteristics of the vehicle, and operatingcharacteristics of other vehicles. According to an exemplary embodiment,scene 100 includes pedestrian 120, lane markers 125 and 130, vehicle 135and roadway boundary 140.

Vehicle 105 may not only identify a type of target object, such aspedestrian or passenger vehicle, but may also utilize one or moreattributes of the object to characterize a target object. Attributes andobject types may be stored in annotated baseline data or vehicle traineddata that may be utilized by a control unit of vehicle 105 to identifyobjects and in some cases control vehicle 105. In an exemplaryembodiment, vehicle 105 may detect lane markers 125 and 130 and one ormore attributes associated with each target. By way of example, vehicle105 may detect the location, spacing, color, angular diversion, and oneor more additional attributes to detect a scenario that may be ofinterest to the vehicle. In some cases, such as highway driving, vehicle105 may be trained to detect and handle lane markers 125 and 130 thatappear in more or less a straight line for an extended period.Identification of these attributes may be used by an assistive system ofvehicle 105 to stay within lane markers 125 and 130. As will bediscussed herein, vector representation of sensed vehicle data fortarget objects, such as lane markers 125 and 130, can be used toidentify scenarios where the trained vehicle data is not equipped tohandle target objects, such as a sudden departure in positioning of lanemarkers 125 and 130, or a narrowing of roadway 140. In addition to eventcapture, configurations are directed to determining when and what typeof vehicle sensor data is not adequately included in vehicle traineddata. As such, configurations are provided for identifying at least oneof a target object of interest, scenario of interest and range of datacollected by a vehicle.

In yet another embodiment, configurations are provided to report one ormore of a target object and scenario of interest. According to oneembodiment, vehicle 105 may be configured to communicate withcommunication network 115 for data exchange with one or more networkresources, such as servers, to share target objects and scenarios. Incertain embodiments network resources may be used to provide updates tovehicle trained data used by vehicle 105 in response to reported objectsor scenarios.

Another embodiment is directed to minimizing the data reported byvehicle 105. Vehicles configured for assisted or autonomous driving mayrecord vast amounts of data. While vehicle data may be recorded andstored in a vehicle recording device to track vehicle operation, theremay be many cases where reporting of vehicle data is unnecessary. Forexample, reporting may not be necessary when the vehicle 105 canadequately handle the scenario or scene based on a trained vehicle dataset. Referring to the highway driving example described above betweenlane markers 125 and 130 for long stretches (e.g., on the order ofmile/km), reporting and later processing of the data and scenario maynot be required. In fact, reporting and processing the data post vehicleoperation may require large data stores and actual manual laborrequiring a financial outlay. In certain situations, there is a need forconfigurations that do not report vehicle sensor data that is adequatelyhandled.

Another embodiment includes identifying target objects or scenarios thatare unknown or not properly classified by a vehicle control unit. By wayof example, vehicle 105 may detect several pedestrians, such aspedestrian 120, or several vehicles, such as vehicle 135, whereinattributes of the vehicle trained data set can account for differencesin the actual objects while still classifying the objects to the correctobject type and/or appropriate object attributes. However, in someinstances, vehicle trained data sets may not accurately or adequatelyidentify an object or scenario. FIG. 1A depicts a significant scenario145 as a question mark, as vehicle sensor data may identify thatscenario 145 is associated with a target object or vehicle conditionthat is not understood using the vehicle trained data set. For example,the trained data set may not understand a target object using thetrained data set when one or more of the object, object attribute, anddriving condition exceed a degree or threshold relative to a vectorrepresentation of the detected sensor data. Operations andconfigurations discussed herein provide for various unidentifiedscenarios.

With respect to target objects, scenario 145 may relate to anunidentified object. Described another way, scenario 145 may relate toan object detected by vehicle 105 but unknown to the vehicle. By way ofexample, if scenario 145 relates to a person dressed in a chicken suit(or other costume for that matter), attributes of the vehicle traineddata set used by vehicle 105 may not be able to identify the object, asthe chicken suit would likely result in characteristics not includingattributes for a pedestrian object type. Or possibly the chicken suitmay be associated with object attributes for two different object typesresulting in a conflict. Another object type example for scenario 145may relate to detection of a rare object, such as a monster truck (e.g.,vehicle with extremely large tires, and possibly pyrotechnics) that isdetected by vehicle 105. Attributes of target object types for vehiclesin a trained vehicle set will typically be directed to passengervehicles, a detected monster truck may be a scenario of interest.Scenarios of interest may be characterized as being worth reporting byvehicle 105. Scenario 145 may relate to vehicle operatingcharacteristics, such as abrupt maneuvering (e.g., sliding, loss oftraction) or vehicle control (e.g., braking). During a typical rush hourdrive a vehicle operator (or autonomous system) may heavily depress thebrakes. In some situations, repeated braking in close proximity to othervehicles may be normal for a particular vehicle during a particular timeof day and/or on a particular route/stretch of road. However, in othersituations, abrupt or repeated braking or other vehicle operatingconditions may be a scenario, such as scenario 145, worth reporting. Byway of example, the vehicle trained data set for assistive braking orautonomous driving may not be trained to maintain a proper distance ofseparation. As such, systems, devices, and vehicle configurations areprovided for identifying scenarios of interest relative to a trainedvehicle data set.

With respect to object types and scenarios, the principles of thedisclosure may also apply to assessing the familiarity of a vehicleassistive driving unit as a whole for various scenarios. Assistivedriving systems trained on roadways in pleasant weather conditions mayhave difficulty when weather or other situations arise. Weather, thepresence of snow or other precipitation and even loss of grip may beattributes accounted for assessing vehicle trained data sets.

FIG. 1B illustrates an exemplary segment of objects relative to atimeline. With respect to reporting, a control unit of a vehicle (e.g.,vehicle 105) may be configured to report at least one of an object ofsignificance and a segment of data captured by the vehicle. In somecases, reporting a target object may allow for the vehicle to betterunderstand unfamiliar pedestrians, such as a man in a chicken suit. Inother instances, reporting all target objects and attributes may be fora period of time preceding and following a target object. FIG. 1B showsscene data 150 relative to a timeline 155 and for a segment 160 ofinterest. Segment 160 may be identified relative to target objects andattributes identified when reporting. As will be discussed in moredetail below, a segment of data, such as vehicle sensor data may bereported in response to a scene detection operation to identify asignificant scenario or object. FIG. 1B is an exemplary representationof segment 160, where objects prior to the segment, such as vehicle 165,will be excluded from the reporting. In one embodiment, reporting asegment of object targets includes reporting objects priori to asignificant scenario data, such as pedestrian 170 prior to, scenario180, and objects following the scenario, such as vehicle 175.Embodiments herein allow for reporting simply scenario 180, and/orreporting segment 160 including objects 170 and 175 and scenario 180.

While FIGS. 1A-1B depict an exemplary scene and segment ofobjects/scenarios that may be detected, embodiments are provided hereinfor identification of significant data and scenarios.

FIG. 2 depicts a process for identifying significant scenario dataaccording to one or more embodiments. Process 200 may be performed by acontrol unit (e.g., control unit 305 of FIG. 3). According to oneembodiment, process 200 includes receiving vehicle sensor data capturedby at least one sensor of the vehicle at block 205, running a scenedetection operation at block 210, and generating a vector representationof a scene at block 215. A significant scenario may be identified atblock 220. Process 200 may be performed by a control unit, such as anavigation control unit, of a vehicle. The control unit may beconfigured to receive positioning data for the vehicle, such as globalpositioning data (e.g., GPS). The control unit may also store map datafor one or more geographic regions associated with vehicle position. Incertain embodiments, the control unit is configured to receive roadwayinformation services such as traffic and weather associated with routesof the map data. Roadway and condition information such as weather maybe included in one or more attributes of a trained vehicle data set.

According to one embodiment, process 200 may be based on a trainedvehicle data set. By way of example, a trained vehicle data set mayinclude an annotated training data set from an advanced driverassistance system (ADAS) or autonomous driving (AD) system with rawsensor data based on several attributes with ground truth. Vehiclesensors can include data for one or more of video, radar, and LiDAR asground truth used for annotation. Ground truth data provides the systemwith a perceptible object types and attributes. Process 200 can allowfor the extraction and use of a driver assistance system operatingparameters (e.g., algorithm, processes, etc.) during runtime to assessfamiliarity with its own baseline data set to identify and flag forcapture an anomaly. This capture would be used to update the training ofthe run time algorithm in a subsequent release by updating the baselinedata set.

At block 205, vehicle sensor data from one or more sensors of thevehicle may be received. By way of example, vehicle sensor data isgenerated by a driver assistance system of a vehicle, and in particularone or more sensors associated with or included in the driver assistancesystem. According to one embodiment, vehicle sensor data includes atleast one of image, radar, and LiDAR data for a detection zone of thedriver assistance system of the vehicle. Vehicle sensor data can alsocharacterize operation of a vehicle and one or more other vehiclesrelative to the vehicle to include data for driving distance relative tothe vehicles, number of vehicles, vehicle type, etc.

At block 210, the control unit runs a scene detection operation on thevehicle sensor data using a trained vehicle data set to identify targetobject attributes of the vehicle sensor data. In one embodiment, runningthe scene detection operation on the vehicle sensor data generates anannotated data set for target objects in real time based on theattributes of the trained vehicle data set. By way of example, the scenedetection operation not only identifies target objects, but alsoperforms operations to perceive the objects using a trained vehicle dataset. The trained vehicle data set can provide a plurality of objecttypes and object attributes. The scene detection operation block 210 cangenerate a dataset characterizing a scene relative to a vehicle.According to one embodiment, the dataset can be approximated usingclustering of the types of data based on the attributes.

According to another embodiment, at block 210, the control unit runs ascene detection operation on the vehicle sensor data to derive a vectorof target object attributes of the vehicle sensor data. In oneembodiment, the vector of target object attributes characterizes a scenerelative to a vehicle including the data generated by at least oneperception sensor.

At block 215, the control unit may be configured to compare a vectorrepresentation for the scene detection operation with a familiarityvector of a trained data set.

At block 215, the control unit can include generating a vectorrepresentation for the scene detection operation and the attributes ofthe vehicle sensor data. According to one embodiment, the vectorrepresentation includes one or more operations to generate a model ofthe data for the scene, such as a vector space model representingobjects in a continuous vector space where similar objects are mapped tonearby points. According to one embodiment, the number of attributes forobjects in the trained vehicle set directly correlates to the number ofdimensions of the generated vector. Generating the vector representationcan include performing a clustering operation for target objects of thevehicle sensor data using the trained vehicle data set to generate avector data model for the vehicle sensor data. For example, a clusteringmethod such as K-means clustering may be used to approximate the “area”of a collected and annotated data set that is used to train a run timealgorithm for scene detection. According to one embodiment, the vectorrepresentation is a representation of effectiveness of the scenedetection operation in identifying target object attributes of thevehicle sensor data. The vector data model characterizing ability of thetrained vehicle set to perceive target objects of the vehicle sensordata.

According to one embodiment, a trained vehicle data set is stored in adata repository, such as a cloud repository, and may be provided to oneor more vehicles (e.g., control units). A resultant vector may begenerated from the clustering to describe the dataset for performingscene operations. In one embodiment, the vector is generatedconcurrently with the algorithm release for the run time system.

According to another embodiment, the vector is then used in the run timesystem to score the scenario based on the attributes of detected targetobjects. Process 200 may optionally include scoring a vectorrepresentation at block 225. According to one embodiment, the controlunit can score a vector representation on the ability of the scenedetection operation to perceive target object attributes of the vehiclesensor data using the trained vehicle data set. A significant scenariomay be identified data based on a score of the vector representationbelow a predetermined threshold.

At block 220, the control unit can identify significant scenario databased on the vector representation. The significant scenario identifiesat least one target object of the vehicle sensor data. According to oneembodiment, an anomaly that escapes the clustering described by thevector would trigger a flag in a data collection system that describesan event. Each event may be considered significant to the existing dataset since the trained run time algorithm is not familiar with thescenario. The compiled events captured during the capture period areadded to the resultant data set and cycled through the training processfor the next algorithm release. The new data set is then approximatedagain to update the clustering.

According to one embodiment, identifying the significant scenarioincludes determining that a target object is an unidentified object.Objects that are unidentified, such as a person in a chicken suit,monster truck vehicle, or unidentified flying object (e.g., drone,flying objects in general) may be classified as unidentified andreported. In some cases, network resources may review the report objecttargets or segments. By providing a report of the unidentified object,the reviewing system can focus resources on objects that areunidentifiable using the trained data set.

According to another embodiment, identifying a significant scenarioincludes determining that at least one of the trained vehicle dataattributes are unable to classify a target object. In this scenario, anobject may or may not be associated with a target object class; howeverone or more features or attributes of the target object may not beclassified by the trained vehicle data set. Accordingly, objectattributes may be added or modified to allow for classifying an objectbased on a new parameter.

According to another embodiment, identifying the significant scenarioincludes determining familiarity of a target object relative to thetrained data set based on at least one of the number of target objects,classification of target objects, size and shape of target object,object type, and object color. In certain embodiments, attributes may bestored in the trained vehicle data set for a plurality of attributesbased on previously detected object targets.

According to another embodiment, identifying the significant scenarioincludes determining at least one vehicle operation characteristic as anattribute relative to identification of a target object in at least oneof a driver assistance system and autonomous driving system. In certainembodiments, attributes are provided in the trained vehicle data set forsensor configurations of the vehicle. In an exemplary embodiment,attributes may be based on a front camera used in vehicle assistancesystem or autonomous vehicle to bring in data into a central computingplatform of the vehicle.

Process 200 may optionally include outputting a target object and/orscenario at block 230. Outputting the significant scenario data caninclude output of an object, multiple objects, and a segment of data. Incertain embodiments, segments of data output may include data for targetobjects detected in a period preceding and following identification of asignificant event. Reporting significant scenario data is an improvementover systems that report all data, especially when a vehicle is operatedfor an extended period of time.

According to an exemplary embodiment, process 200 may includeidentifying significant scenario data based on a vector representationof a trained data set and vehicle sensor data from one or more sensors(e.g., sensors 335 _(1-n) of FIG. 3). Receiving vehicle sensor data atblock 205 can include receiving data from vehicle sensors including butnot limited to an image sensor (e.g., video), radar sensor, LiDARsensor, acceleration/motion sensor and vehicle sensors (e.g., wheelspeed sensor, tire pressure monitoring system (TPMS) sensor, ABS sensor,yaw/pitch sensor, stability control sensor, etc.). Sensor data may bereceived by a control unit by way of a controller area network (CAN) busof the vehicle. Sensor data received at block 205 may be used to run ascene detection operation at block 210.

In one embodiment, the scene detection operation at block 210 identifiesreceived sensor data and a vector representation may be generated forsensors individually and/or in combination. In certain embodiments, ascene detection operation may be performed to identify sensor data thatis outside of a normal or trained operating range. For vehicle handling,for example, one or more of throttle sensor output, braking controls(e.g., ABS braking system sensors), tire pressure, etc., may be detectedfor a period of time. When one or more of the sensors have a change inoutput associated with an event sensor data preceding, during and afterthe event may be captured. A significant scenario may be detected atblock 220 when one or more sensor outputs exceed or differ from atrained data set. By way of example, for a vehicle traveling on at highspeeds during highway driving, the scene detection operation may includetrained data for a particular vehicle speed with braking, tire pressureand other sensor data to be in a relative range. When sensor dataindicates a departure from expected bounds for the scenario, such asvehicle a being controlled or operated at speeds too high for exitingthe highway as indicated by one or more of elevated tire pressure,increased breaking, vehicle sensor data may indicate a significantscenario where trained data does not provide acceptable operation of thevehicle. Because highway exits often differ, trained data for a vehiclemay benefit from scenes and scenarios captured by the vehicle.

According to another embodiment, sensor data detected at block 210 mayresult in a significant scenario when the sensor does not provide anaccurate representation. By way of example, one or more sensors of thevehicle may be configured for certain operating parameters, such as acameras definition based on the number of pixels or frames detected. Forcertain driving scenarios, such as low speed driving, the cameraresolution and frame rate may be acceptable for driving at low speeds,while higher speed operations may require increase resolution or ratesof detection. Accordingly, a significant scenario may be detected atblock 220 may relate to an indication of the sensors ability to providedata in addition one or more training parameters for the vehicle controlunit. Outputting a scenario at block 230 can include providing sensordata in addition to the indication of the event. For camera devices,outputting scenario at block 230 can include transmitting image datacaptured and objects identified by the vehicle control unit as part ofthe scene detection operation.

According to one embodiment, when the scenario relates to a pattern ofdetected data, event significance may be related to departures inpatterns or pattern types that are not trained. According to oneembodiment, vehicle sensor data at block 205 relates to a pattern, suchas roadway markers, or road width. Patterns may also relate to objectsrelative to a road, such as barriers or bicyclists. The scene detectionoperation at block 210 may detect the pattern from sensor data and avector representation of the sensor data may be employed to characterizeoperation of the vehicle relative to the pattern. Significant scenariosmay be detected at block 220 when received sensor data of the patternchanges. By way of example, roadway markers for a period of time, suchas 100 yards (e.g., 300 m) that suddenly change or result in a laneshift. Similar examples include the presence of lane barriers and then ashift in position. Another example could be the sudden appearance of alane barrier. With respect to movable or moving objects a significantscenario may be operating characteristics of another vehicle thatappears to be swerving. Another example of the significant scenario maybe the pattern of a motorcycle or bicyclist traveling in apattern/expected trajectory that sudden shifts toward a vehicle of thecontrol unit. In such an instance, one or more vehicle systems such as astability control modules may generate a scenario that is not recognizedand/or not trained.

According to one embodiment, process 200 may include identifying asignificant scenario by determining familiarity for a current scenevehicle is operating in relative to a vehicle trained data set for atleast one driving condition. A significant scenario may be detected atblock 220 by characterizing when received sensor data of the patternchanges.

FIG. 3 depicts a graphical representation of a vehicle control unitaccording to one or more embodiments. According to one embodiment, avehicle includes a control unit 305 which may be configured to interfacewith one or more vehicle components. According to one embodiment,control unit 305 may be configured to perform one or more processes andfunctions described herein, such as process 200 of FIG. 2. Control unit305 may relate to a control unit of a vehicle navigation unit, advanceddriver assistance system (ADAS) or autonomous driving (AD) system.

In an exemplary embodiment, control unit 305 includes one or moreelements or modules for interfacing with vehicle components. FIG. 3shows control unit 305 including a control module 306, scene detectionmodule 307 and vector generation module 308. Control unit 305 mayreceive position data for a vehicle from receiver 310. One or moreexecutable instructions and navigation data (e.g., map data) may bestored by data storage 320. Input output module 320 may be configured tointerface one or more other devices including but not limited to networkresources. Control unit 305 may be configured to communicate with avehicle system 330, including an engine control unit (ECU).

According to one embodiment, control module 306 represents one or morefunctional and hardware elements of control unit 305 that may beconfigured to direct operation of the control unit. Control module 306may be configured to receive and utilize a trained vehicle data set.Control module 306 may direct one or more communications to vehiclesystem 330, which may include output to one or more of a vehicle bus andelectronic control unit configured to control vehicle operation (e.g.,braking, lighted indicators, safety features, etc.).

According to one embodiment, scene detection module 307 represents oneor more functional and hardware elements of control unit 305 that may beconfigured to analyze target objects and vehicle operations. Scenedetection module 307 may identify one or more significant events.

According to one embodiment, vector generation module 308 represents oneor more functional and hardware elements of control unit 305 that may beconfigured to generate a vector and one or more anomalies against thevector. Vector generation module 308 may estimate the familiarity of atrained vehicle set with a current driving scene or environment. Sensormodule 308 may also receive one or more control signals for a sensorthat is part of vehicle system 330 relating to the status of a filter.

According to one embodiment, the principles, processes and deviceconfigurations may be applied to one or more sensor packages. FIG. 3 isshown including sensors 335 _(1-n) configured to provide data to vehiclesystem 330. According to one embodiment, sensors 335 _(1-n) may relateto one or more sensors of a vehicle associated with control unit 305.Sensors 335 _(1-n) may provide output to a CAN bus of vehicle system 330which may also be received by control unit 305 by way of a CAN bus orother vehicle system.

Sensors of the vehicle can include one or more of an image sensor (e.g.,video), radar sensor, LiDAR sensor, acceleration/motion sensor andvehicle sensors (e.g., wheel speed sensor, tire pressure monitoringsystem (TPMS) sensor, ABS sensor, yaw/pitch sensor, stability controlsensor, etc.), throttle sensor, and vehicle sensor in general. Output ofsensors 335 _(1-n) may be provided as vehicle sensor data to controlunit 305 for processing.

According to one embodiment, sensors 335 _(1-n) may relate to one ormore vehicle acceleration sensors and/or vehicle stability controlsensors (e.g., traction control). Sensors 335 _(1-n) may include aplurality of acceleration sensors (e.g., accelerometer) and sensors foryaw, pitch and steering angle. In certain embodiments, identifying asignificant event may be related to the ability of a vehicle traineddata set to control vehicle operation relative to a roadway or curve. Assuch, scoring may generate an objective value representing the abilityof a traction control system of the vehicle to handle a roadway. Forexample, sensors 335 _(1-n) may generate output indicating understeer oroversteer from one or more of sensors 335 _(1-n). Sensors 335 _(1-n) mayinclude a vehicle accelerometer (e.g., single axis, multi-axis) and oneor more yaw/pitch sensors to track vehicle displacement through a curve.In situations where the trained data set is used to operate the vehicleand/or employed by the traction control system, a significant event maybe determined for understeer and/or oversteer.

FIG. 4 depicts a graphical representation of a trained vehicle data setaccording to one or more embodiments. According to one embodiment, atrained vehicle data set is a known training data set used to trainalgorithms that operate autonomous vehicles and assisted driving systemsof various types. FIG. 4 provides an exemplary illustration of a trainedvehicle data set relative to an ideal data set.

According to one embodiment, an ideal data set can be reached where mostor all driving scenes are adequately handled. The ideal data set for avehicle may depend on the sensors of the vehicle, sensing ability,sensor data, etc. Even with improved sensors, there is a need to assessthe ability of a system to detect and handle objects and scenarios. Inaddition, there is a need to identify situations a vehicle training dataset does not adequately address.

FIG. 4 illustrates a representation of a known or existing data set 405relative to an ideal data set 410. Existing data set 405 may relate to aknown data set including a known training data set used to trainalgorithms that operate autonomous vehicles of various types. Processesand operations discussed herein may generate a familiarity metric orvector, shown as vectors 415, using data captured by one or more localvehicle sensors or combined sensors, and subsequent real-time algorithmsmeasuring scene statistics. In one embodiment, the familiarity of dataset 405 is generated based on many possible attributes captured by thehost vehicle and associated algorithms. According to one embodiment,exemplary attributes can include, but are not limited to: number oftarget objects, classification of target objects, size and shape oftarget objects, number of lanes, lane type, lane marker color, lanemarker shape or dash type, vehicle state behavior such as sliding,vehicle location, environmental conditions that effect visibility oftarget objects and lanes. These attributes may be employed to describethe dimensions of a vector, or vectors 415, generated by a neuralnetwork that uses a technique of clustering known attributes of a knowntraining data set. One or more vectors, such as vectors 415, may be usedin a run time environment to detect anomalies against the vector. Thedescribed vector is the estimate of the run time algorithm's familiaritywith the environment in which it is used. To describe the general use ofthe aforementioned vector, the vector is generated as part of a softwarerelease for an autonomous vehicle control system that describes theentire training set of data. In one embodiment, the vector is scoredcontinually during run time in the autonomous vehicle control system aspart of the perception engine. This scoring provides the method toidentify significant or unfamiliar scenario data and flag or recordassociated data in a previously unmentioned data recorder.

Ideal data set 410 may represent a set of data that may not be achieved.Systems and configurations discussed herein may operate based on dataset 405 that is known. An anomaly detection vector may be generated fordata set 405 to describe the data set. Processes discussed hereinprovide a trigger mechanism that triggers an event or scenario that hasnot been presented in the data set 410.

FIG. 5 depicts a graphical representation of control unit operationsaccording to one or more embodiments. Control unit operations canfacilitate the automatic identification of new data that an existingassistive vehicle algorithm would be unfamiliar with. According to oneembodiment, a vehicle control unit 500 may perform functions for atleast one of significance anomaly detection and validation anomalydetection. Vehicle control unit 500 may be configured to receive atrained vehicle data set 501 from a data repository, such as a cloudrepository. The vehicle data set may be provided to a vehicle to performscene detection and/or other operations.

In one embodiment, vehicle control unit 500 includes a significanceanomaly detection engine module 505 configured to perform a clusteringmethod, such as K-means clustering, to approximate the “area” of thecollected and annotated data set that is used to train the run timealgorithm. A resultant vector, from anomaly vector block 510, isgenerated from clustering and is used to describe the dataset. Thevector may be generated concurrently with the algorithm release for therun time system. In one embodiment, the vector is then used in a runtime system to score the scenario based on one or more attributes. Ananomaly that escapes the clustering described by the anomaly vectorwould trigger a flag at event trigger block 515 in a data collectionsystem that describes an event. According to one embodiment, this eventis significant to the existing data set 501 since the trained run timealgorithm is not familiar with the scenario.

According to one embodiment, compiled events captured during the captureperiod are added to the resultant data set and cycled through thetraining process for the next algorithm release. The new data set isthen approximated again to update the clustering of objects.

At block 520, validation anomaly detection engine receives the anomalyvector from block 510. According to one embodiment, the anomaly vectormay be used by validation anomaly detection engine module 520 to assessthe familiarity of data set 501 within instantaneous analysis of ascene. Validation anomaly detection engine module 520 can detectanomalies against the anomaly vector from block 510 and generate areport on the familiarity of the data set 501 with a scene, scenario, orvehicle sensor data.

FIG. 6 depicts a graphical representation of operations relative to aknown data set according to one or more embodiments. Process 600describes operations for a vehicle system according to one or moreembodiments. In one embodiment, process 600 is directed to performingoperations for significance anomaly detection. According to anotherembodiment, process 600 is directed to validation of an anomalydetection engine. Process 600 may relate to one or more control unitoperations of a significance anomaly detection engine module and avalidation anomaly detection engine module (e.g., significance anomalydetection engine module 505 and validation anomaly detection enginemodule 520 of FIG. 5).

According to one embodiment, baseline data set 605 including a pluralityof object types and object attributes may be employed as a knowntraining data to train algorithms of a vehicle assistance system, suchas a scene detection operation. A clustering quantization operation isperformed at block 610 for the baseline data set and a vector generationoperation is performed at block 615 based on characteristics of the dataclusters generated at block 610. The clustering quantization may includeone or more operations to classify and annotate objects by type andattributes. As a result of clustering, objects having similar attributesmay form clusters that may be analyzed by vector based modeling. Given aknown data set, such as baseline data set 605, including a knowntraining data set used to train algorithms that operate autonomousvehicles of various types, process 600 is configured to generate afamiliarity metric or vector at block 615. One or more vectors may begenerated describing the centroid of each cluster form block 610 andresult in a representation that may be employed by a vehicle controlunit to analyzed sensed data and/or the baseline data set 605.Operations at block 610 and 615 may be performed to describe thebaseline annotated data set 605, such as whether or not the baselinedata set includes data similar or capable of describing sensed vehicledata. The vector generated at block 615 may be used to identify objecttargets or vehicle operating conditions of significance. According toanother embodiment, blocks 610 and 615 may be embodied as computerexecutable code that may be deployed to a vehicle for use in a driverassistance system. According to one embodiment, attributes are used todescribe the dimensions of a vector generated at block 620 by a neuralnetwork using clustering of the known attributes at block 610 thetraining data set 605.

According to another embodiment, operations for anomaly detection may beprovided by process 600. Using data captured by a local vehicle sensoror combined sensors at block 606 and subsequent real-time algorithmsmeasuring scene statistics such as a perception algorithm at block 625,familiarity of the baseline data set may be determined. Process 600includes scoring attributes against an anomaly vector at block 630.Scoring may provide one or more values, such as a percentage or valuewithin a range identifying characterizing familiarity of the baselineannotated data set 605 with objects detected at block 625. Scoring maybe based on many possible attributes captured by the host vehicleprovided in sensor data 606 and associated algorithms performed at block625. Exemplary attributes in one embodiment can include, but are notlimited to: number of target objects, classification of target objects,size and shape of target objects, number of lanes, lane type, lanemarker color, lane marker shape or dash type, vehicle state behaviorsuch as sliding, vehicle location, environmental conditions that effectvisibility of target objects and lanes. Scoring at block 630 can allowfor identification of significant or unfamiliar scenario data to beflagged or record associated data in a data recorder or storage unit ofa vehicle.

According to one embodiment, scoring at block 630 is performed based ona predetermined amount of variability. According to one embodiment, forobjects detected optically, an acceptable amount of variability may beallowed for certain object classifications. Accordingly, scoring basedon a first object class may employ a first amount of variability, whilescoring a second object type may employ a second amount of variability,the second amount of variability different from the first. Thus, camerasensor data from block 606 identifying a pedestrian object may have alow threshold for determining significance, relative to signage which isusually associated with a limited set of pre-established set of objectsand thus, have a higher threshold for significance. With respect toscoring sensor attributes, scoring may asses the divergence of sensoroutput relative to trained sensor ranges. In certain embodiments,scoring at block 630 is configured to assess a trained data sets abilityto perceive a driving scenario based on sensor data 606. By way ofexample, a trained data set may provide a basis for one or more ofpresenting an indication, controlling a vehicle unit or activating avehicle control system (e.g., active breaking, throttle control, etc.)based on sensor data 606 and vector generation 616. Scoring at block 630can provide an indication of the trained data sets ability to handlesensed data. In one embodiment, scoring at block 630 can result in atleast one of a percentage and/or value measurement of accuracy for thetrained data set relative to sensor data 606. In certain embodiments,the determined score may be output with generated flags in order forupdating a trained data set.

According to one embodiment, operations performed by control unit 620employ a vector generated at block 615 in a run time environment todetect anomalies against the vector. The described vector is theestimate of the run time algorithm's familiarity with the environment inwhich it is used. To describe the general use of the aforementionedvector, the vector is generated as part of a software release for anautonomous vehicle control system that describes the entire training setof data. The vector is scored continually during run time in theautonomous vehicle control system as part of the perception engine.

At decision block 635, process 600 determines whether target objectsthat are detected are part of a baseline data set, such as baselineannotated data set 605. When objects are included in the dataset,scoring at block 630 can reflect the data sets familiarity with sensedtarget object. When the target objects are not part of the baseline dataset, an event flag to identify a target object or an event capture todetect a segment of target objects may be generated at block 640.Process 600 may include outputting event flags and event capture ofblock 640 to a data storage unit 645 or network resource (e.g., server,cloud network) which may be used to annotate and/or update a baselineannotated data set 605. Event flags and event captures in data storage645 may also be used to identify one or more of objects, objectattributes and scenarios that are unique.

FIG. 7 depicts a graphical representation of object attributes accordingto one or more embodiments. According to one embodiment, a baseline dataset may be characterized by attributes associated with object typesand/or object characteristics. Process 700 describes clustering ofobject attributes by a vehicle control unit according to one or moreembodiments.

Process 700 may be based on a plurality of attributes, such asattributes 705 _(1-n). Attributes of process 700 include number ofobjects 705 ₁, object class 705 ₂, size and shape (e.g., dimension) 7053and color 705 _(n). It should be appreciated that other attributes maybe employed. Attributes 705 _(1-n) may also be based on the sensors of avehicle. Using attributes 705 _(1-n) a vector representation of a scenemay be generated at block 710. The vector representation may be based ona vector data model representing detected objects by attributes, such as705 _(1-n), in a plurality of dimensions. The vector representation maybe based on clusters of data representing similar types of objects. Datarepresentation 715 represents an exemplary representation of dataincluding a cluster 720 associated with a baseline data set. Cluster 720represents data that is known, whereas data point 725 represents ananomaly or object target that is not similar to data point 725. Datapoint 725 is referred to as a point for illustration of a detectedobject or scene that lies beyond the object, object attributes and orscene parameters of the trained data set represented by cluster 720.According to one embedment, an anomaly vector determined for cluster 720may be employed to identify data point 725.

According to one embodiment, the vector representation of a scene may begenerated at block 710 using attributes 705 _(1-n) may relate to avector model representation a trained data sets ability to perceive andhandle a detected object and/or scene. According to one embodiment,identifying a significant event may be based on the amount of divergenceof a data point, such as data point 725, from a cluster, such as cluster720, of the vector data model. As discussed herein, scoring ofattributes may result in a value or representation of data pointdivergence from a cluster, such as cluster 720.

While this disclosure has been particularly shown and described withreferences to exemplary embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the scope of the claimedembodiments.

1. A vehicle control unit comprising: an input configured to receivevehicle sensor data; and a control unit coupled to the input, whereinthe control unit is configured to: receive vehicle sensor data capturedby at least one sensor of the vehicle, the vehicle sensor data generatedby at least one perception sensor of a vehicle; run a scene detectionoperation on the vehicle sensor data to derive a vector of target objectattributes of the vehicle sensor data; compare the vector of targetobject attributes against a cluster of a vector data model of a trainedvehicle data set; and identify significant scenario data based on adivergence between the vector of target object attributes and thecluster, wherein the significant scenario data identifies at least onetarget object of the vehicle sensor data.
 2. The vehicle control unit ofclaim 1, wherein the vehicle sensor data includes at least one of image,radar, and LiDAR data for a detection zone of a driver assistance systemof the vehicle.
 3. The vehicle control unit of claim 1, wherein runningthe scene detection operation on the vehicle sensor data generates anannotated data set for target objects in real time based on theattributes of the trained vehicle data set, the trained vehicle data setproviding a plurality of object types and object attributes.
 4. Thevehicle control unit of claim 1, wherein the cluster of the vector datamodel of the trained vehicle data set is generated by a k-meansclustering algorithm.
 5. The vehicle control unit of claim 1, whereinidentifying the significant scenario includes determining that a targetobject is not represented in the trained vehicle data set.
 6. Thevehicle control unit of claim 1, wherein identifying the significantscenario includes determining that the trained vehicle data set isunable to classify a target object.
 7. The vehicle control unit of claim1, wherein identifying the significant scenario includes determiningfamiliarity of a target object relative to the trained data set based onat least one of a number of target objects, classification of targetobjects, size and shape of target object, object type, and object color.8. The vehicle control unit of claim 1, wherein identifying thesignificant scenario includes determining at least one vehicle operationcharacteristic as an attribute relative to identification of a targetobject in at least one of a driver assistance system and autonomousdriving system.
 9. The vehicle control unit of claim 1, whereinidentifying the significant scenario includes determining familiarityfor a current scene of vehicle operation relative to a vehicle traineddata set for at least one driving condition.
 10. The vehicle controlunit of claim 1, wherein the control unit is further configured to scorethe vector on an ability of the scene detection operation to perceivetarget object attributes of the vehicle sensor data using the trainedvehicle data set, and wherein the significant scenario is identified, bythe control unit, based on a score of the vector of target objectattributes being below a predetermined threshold.
 11. (canceled)
 12. Amethod for identifying significant scenario data by a control unit of avehicle, the method comprising: receiving, by a control unit, vehiclesensor data captured by at least one sensor of the vehicle, the vehiclesensor data generated by at least one perception sensor of a vehicle;running, by the control unit, a scene detection operation on the vehiclesensor data to derive a vector of target object attributes of thevehicle sensor data; comparing, by the control unit, a vectorrepresentation for the scene detection operation with a familiarityvector of a trained vehicle data set, wherein the vector representationis a representation of effectiveness of the scene detection operation inidentifying target object attributes of the vehicle sensor data;identifying, by the control unit, significant scenario data based on thevector representation, wherein the significant scenario identifies atleast one target object of the vehicle sensor data; adding, by thecontrol unit, the significant scenario data to a resultant data set; andupdating, by the control unit, a clustering of the trained vehicle dataset based on the resultant data set.
 13. The method of claim 12, whereinthe vehicle sensor data includes at least one of image, radar, and LiDARdata for a detection zone of a driver assistance system of the vehicle.14. The method of claim 12, wherein running the scene detectionoperation on the vehicle sensor data generates an annotated data set fortarget objects in real time based on the attributes of a trained vehicledata set, the trained vehicle data set providing a plurality of objecttypes and object attributes.
 15. The method of claim 12, whereincomparing the vector representation includes performing a clusteringoperation for target objects of the vehicle sensor data using thetrained vehicle data set to generate a vector data model for the vehiclesensor data, the vector data model characterizing ability of the trainedvehicle set to perceive target objects of the vehicle sensor data. 16.The method of claim 12, wherein identifying the significant scenarioincludes determining that a target object is an unidentified object. 17.The method of claim 12, wherein identifying the significant scenarioincludes determining that the trained vehicle data set is unable toclassify a target object.
 18. The method of claim 12, whereinidentifying the significant scenario includes determining familiarity ofa target object relative to the trained data set based on at least oneof a number of target objects, classification of target objects, sizeand shape of target object, object type, and object color.
 19. Themethod of claim 12, wherein identifying the significant scenarioincludes determining at least one vehicle operation characteristic as anattribute relative to identification of a target object in at least oneof a driver assistance system and autonomous driving system.
 20. Themethod of claim 12, further comprising scoring, by the control unit, thevector representation on an ability of the scene detection operation toperceive target object attributes of the vehicle sensor data using thetrained vehicle data set, and wherein the significant scenario isidentified by the control unit based on a score of the vectorrepresentation being below a predetermined threshold.
 21. The method ofclaim 1, further comprising updating, by the control unit, a training ofa run time algorithm in a subsequent release by updating a baseline dataset.